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Abstract 

The low temperature phase of discontinuous mean-field spin glasses is characterized by the ap- 
pearance of an exponential number of metastable states. Which ones among these states dominate 
the out-of-equilibrium dynamics of these systems? 

In order to answer this question, we compare high-precision numerical simulations of a diluted 
p-spin model with a cavity computation of the threshold energy. Our main conclusion is that the 
aging dynamics is dominated by different layers of metastable states depending on the cooling 
schedule. In order to perform our analysis, we present a method for computing the marginality 
condition of diluted spin glasses at non-zero temperature. 
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I. INTRODUCTION AND RESULTS 



Upon cooling, glass-forming liquids show a dramatic slowing-down of their relaxational 
dynamics. In mean-field models, this phenomenon is caricatured by a weak ergodicity break- 
ing phase transition (the so called "dynamic phase transition" ) which occurs at some critical 
temperature Td [l|, Q|- Surprisingly enough, this phenomenon could be relevant j_n^nder- 



a 



The 



standing the behavior of local search algorithm for hard computational problems 
time-complexity of these algorithms suddenly explodes when some macroscopic parameter 
describing the instances crosses a critical value. 

Despite their apparent simplicity, the dynamics of mean-field models is still poorly under- 
stood in many aspects. Consider, for instance, the following couple of Gedanken experiments. 
In the first one, the system is at infinite temperature at time t = 0: T(0) = oo, and is sud- 
denly quenched below the dynamic transition: T{t) = T < for t > 0. At some time 
tf we measure the observable O and the experiment is finished. We are interested in the 
asymptotic behavior: 

(0)qucnch= lim(0(tf))T. (1) 

tf — *oo 

In the second experiment we choose a cooling schedule, i.e. a smooth function Tsch(T), 
< r < 1, with Tsch(O) = oo and Tsch(l) = T. We start from T(0) = oo and slowly cool 
down the system, keeping it at temperature T(t) = Tsch(t/tf) for < t < t{. Then we 
measure the same observable O and consider the limit 

(O)eooi = lim (0(tf))T..,(iAf) • (2) 
In this paper we address the question: 

(C')quench = (C')cool • (3) 



Notice that both in Eqs. (^J) and the thermodynamic limit is assumed to be taken before 
the long-time limit. In the opposite case the answer would be trivially positive. 

It turns out that the two quantities are in general different: (C) quench 7^ (C^)cooi- In the 
following we shall focus on the internal energy, O = 'H{(y) and we will show that in this case 
(H(cr))quench > ('^^(o"))cooi strictly. This result is quite surprising. The bulk of analytical 



results in t 
model 



lis field comes, in fact, from the solution of the fully connected p-spin spherical 



In this case (C) quench = (C) 



cool- 



Like other properties of the out-of-equihbrium dynamics of mean-field models, the result 
(C*) quench 7^ (C')cooi Can be interpreted in connection with the structure of metastable states. 
It is worth recalling this connection here. Below th e Boltzmann measure splits into an 



exponential number of metastable states. Formally 

(0)eq^5^^.(0)„. (4) 
a 

Such a splitting has been proven rigorously for some completely connected p-spin models jsj. 
Two different states have different values of the extensive observables (energy, magnetization, 
etc). One can therefore construct constrained Boltzmann measures using only those states 
which have a definite value of one of these observables. For instance one can constrain on 
the energy 

(O)eOC (^)- (5) 

It is a simple property of mean field models that purely relaxation dynamics does not nec- 
essarily converge to (Boltzmann) equilibrium. Consider for instance a Curie Weiss model at 
low temperature in a small positive magnetic field. If the dynamics is initiated from a (suf- 
ficiently) negative magnetization configuration, it will converge to a negative magnetization 
state unless one waits a time exponential in the size of the system j^. 

Unlike in the Curie Weiss model, in glassy systems it is difficult to select a particular 
metastable state by switching on a field in the appropriate direction. In general, the ap- 
propriate out-of-equilibrium measure will rather resemble a mixture of metastable states. 
A fascinating hypothesis on mean-field spin glasses is that relaxation dynamics converges 
indeed (as long as one-time observables are concerned) to a constrained measure of the 
form If one accepts this hypothesis, the open question is: which of the many possible 
constrained measures does the dynamics converge to? Our results imply that the answer 
depends upon the path along which the system is driven into its low-temperature phase. 
Viceversa, this path can be tailored in such a way to select different constrained measures. 

Figure n sketches the mechanism for selecting metastable states at different energy densi- 
ties by changing the cooling schedule. The evolution of the energy density and its asymptotic 
value are traced. In the inset we show the complexity (i.e. the logarithm of the number 
of metastable states) at the final temperature. It turns out that cooling slowly the system 
through the dynamic transition one is able to avoid the higher energy (and most numerous) 





thermodynamical energy 



FIG. 1: A pictorial view of the temperature evolution of metastable states in a discontinuous 
mean-field model, without temperature chaos. The gray areas correspond to those regions in the 
temperature-energy plane, such that an exponential number of metastable states is present, within 
a IRSB calculation. Dark gray corresponds to states which are stable with respect to FRSB, while 
light gray corresponds to unstable states. On the left we depict the trajectory of a quench and a 
cooling protocols. On the right (inset) we plot the IRSB complexity (i.e. the entropy of metastable 
states) at a particular temperature. By 'threshold states', we mean here the highest energy states 
such that the complexity is strictly positive. In general FRSB must be used to describe these 
states, as well as to calculate the iso-complexity energy in the light gray region (dashed line). 



metastable states. This is not the case after a rapid quench. We also show the marginal 
energy density which correspond to the appearance of a full hierarchy of 'nested' decom- 
positions of the form In spin glass jargon, this corresponds to full replica symmetry 
breaking (FRSB). Let us notice that these sketches, as well as the examples in the following, 
refer to cases in which a thermodynamic FRSB transition does not exist (in other words the 
marginal energy line does not cross the thermodynamical energy line). This is not, however, 
a fundamental restriction of the theory. 

Already Crisanti, Horner and Sommers noticed that the asymptotic value of one-time 
observables depends upon the cooling schedule for the spherical p-spin model in non-zero 
magnetic field. This phenomenon was further investigated in a non-homogeneous spherical 
model by Barrat, Franz and Parisi jlfll . The crucial point is that the p-spin spheri- 
cal model is an exceptional far as the structure of high-lying metastable states is 
considered. 

Although several conclusions of agree with our findings, they relied on some un- 
controlled assumptions and interpretations. The generality of their results wasn't fully 



appreciated. In this paper we revisit the same problem in a context where numeri- 
cal simulations can be carefully compared with analytical results, and assumptions can be 
checked. Moreover, the model considered is a close (although simpler) relative of a large 
family of hard optimization problems. 

Our conclusions are based on an extensive numerical study of a diluted Ising spin glass 
with p-spin interactions. The dynamics of these models can be simulated with a smaller 
computational effort with respect to their fully-connected counterparts. Moreover, the anal- 
ysis of their static properties has being rapidly developing in the last few years. This paper 
contains one further step in this direction. Developing the ideas of Ref. [l^, we show how to 
compute the marginality condition of one-step replica-symmetry breaking (IRSB) solutions 
for diluted models at non-zero temperature. 

The paper is organized as follows. In Section |n] we define the family of models to be 
investigated and give some anal5d;ical results concerning the metastable states of these mod- 
els. Moreover we explain how the marginality condition is computed. In Sec. Illll we present 
our numerical simulations and compare them with the static calculations mentioned above. 
A few comments on these results are presented in Sec. IIVI In Appendix^ we collect some 
explicit formulae for the model of Sec. |nj Finally in the App. El we show that, in the large 
connectivity lirnit, our computation of the marginality condition coincides with the usual 
replicon result [13[. 

II. THE MODEL AND ITS MARGINALITY CONDITION 

We shall consider the family of diluted spin glasses defined by the Hamiltonian 

^(^) = - ^n-V^n • • • • (6) 

{ii...ip)eg 

where Q is the hypergraph of interactions, i.e. a set of M among the (^) possible p-uples 
(with p > 3) of the spins. We shall consider hypergraphs Q with fixed connectivity: each 
spin is supposed to participate to c = / + ! interaction terms (hereafter c > 3). The graph 
Q is drawn randomly, with uniform probability, among all such hypergraphs (i.e. among all 
the hypergraphs on A^ vertices with connectivity c, and p- vertices edges). Finally we shall 
assume the couplings Jii...ip to take the values ±1 with equal probabihty. 



This model was already studied in Refs. 



15l | within a IRSB approximation. As the 
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TABLE I: Critical temperatures for the static (Tg) and dynamic (Td) transitions of the model 
© as obtained from a population dynamics solution of the IRSB equations. We also report the 
complexity (Sd) and energy (cd) of threshold states at the dynamic transition. Quoted errors 
include both statistical and systematic uncertainties. The two central columns refer to parameter 
values used in the numerical cooling experiments, while the last column contains the temperatures 
used for the quenching experiments. 

temperature is lowered it undergoes a dynamic phase transition at Td > and, if c > p, 
a static phase transition at Ts, with < Ts < Td. For I > Ig{p) a second static phase 
transition takes place at some temperature Tq: the T < Tq phase is characterized by full 
replica-symmetry breaking. In Ref. we studied the T = limit and found /g(3) = 10. 
In Tab. Uwe report the values of Ts and Td for p = 3 and a few values of / < 10. 

At a fixed temperature T = l/j3 < Td, the Gibbs measure decomposes over an exponential 
number of pure (metastable) states. The number of states at a given free energy density / is 
given, at the leading exponential order, hj Af^{f) ~ exp{A^S^(/)}. In IRSB approximation, 
the complexity T^pif) is obtained as the Legendre transform of the m replicas free energy 
m(f){m,(3) with respect to the parameter m: 



(7) 



The parameter m can be varied in the range md{T) < m < ms{T). This corresponds to 
selecting states of free-energy densities /d(T) > / > fs(T). Any other observable can be 
computed over metastable states of a given free energy, by properly tuning m. The internal 
energy, for instance, decreases from ed(T) to es(T), as / varies from /d(T) to /s(T). In 
Tab. miwe report the curves es(T) and ed(T) for p = 3 and a few values of /, see also Figs. |2l 
121 The IRSB saddle point equations were solved using the population dynamics algorithm 
of Ref. l3. 



As shown in Ref. 



12| |. high-energy metastable states are, quite generically, unstable to- 



wards FRSB. Here we want to compute the stability threshold uiQiT) and the corresponding 

6 



internal energy edT) at finite temperature T. We proceed by determining the instability 
toward two-steps replica symmetry breaking (2RSB), which in turn is expected to imply the 
instability towards FRSB. We shall present the method in a general setting and give some 
numerical results, cf. Tab. IHI We refer to App. ^ for explicit formulae in the case of the 
Hamiltonian 

Let S be the space of normalized measures p{x) over the real numbers. The 2RSB order 
parameter for the model © is a probability distribution Q[p] over S. This is slightly simpler 
than the most general 2RSB order parameter for a diluted model because we assumed all 
the sites to be equivalent Q]. Notice in fact the finite neighborhood of any two sites i and 
j are identical (up to a gauge transformation cij — > TjCTj) in the thermodynamic limit. 

We consider now a generic model such that, once an interaction term has been taken 
away, each adjacent spin interacts with k other ones [to make contact with the Hamiltonian 
(0) one should take k = l{p — 1)]. The 2RSB saddle point equations for such a model have 
the form 

k 

Q^p'^ = ^ [ Wqip^i ^[pi' • • • ' pf^r'^""' ^ ip - po[pi' • • • ' pi^ii ' (8) 
i=i 

where mi < m2 are the two breaking parameters required for 2RSB The functional 

Po[- • • ] reads 

1 f 

Po{x) = — / TT(ipi(xi) w(xi, . . . (5(x - /(xi, . . . ,Xfe)) , (9) 

z[pi, . . .,pk\ J fj^ 



and z[pi,...,pk] is fixed by the normalization condition Jdxp.{x) = 1. The functions 
w{xi, . . . , Xk) and f{xi, . . . , Xk) are model-dependent. Their explicit form for the model (jH} 
are reported in App. |X1 

An IRSB solution is recovered if Q[p] is supported on (5-functions. We want to compute 
the stability of this subspace under the recursion (jHJ. We consider therefore an order pa- 
rameter Q[p] which is concentrated on narrow distributions p{x). Such distributions can be 
characterized by their first two moments: 

2 



X 



J dpi.)., ../dpWx^-(/dpWx) . (10) 



In the e — ^ limit we can derive from Eq. (jH} a recursion for the probability distribution 
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Q{x, e) of these two parameters: 

Q{x,e) ~ i / JJc?Q(^i,ei) wixi, . . . ,Xk)"^^ s(x- f{xi, . . . (^(e - X] i^^f)'^ ^i)^^^) 

1=1 i=l 

Notice that the largest breaking parameter m2 plays no role in the relation pip . Hereafter 
we shall drop the subscript in the remaining parameter and set m = mi. Let us call p*(a:) 
the marginal distribution of x: p*{x) = JdeQ(x,e). It is straightforward to see that this 
distribution satisfies the IRSB equation: 

p*{x) = — JJrfp*(xi) w(xi, . . . 6(w- f(xi, . . .,Xk)^ . (12) 

* i=l 

Therefore p*(x) is nothing but the IRSB solution with parameter m. If Q(x, e) p^(x)6{e) 
under the iteration (fTT|) . the IRSB space is stable. In the opposite case, it is unstable 
towards 2RSB and (presumably) FRSB. 

The distribution Q{x,e) can be represented by a population of couples {(xi,ei) : i = 
1, . . . , A/"}, and the recursion (ITTl) can be approximated by a population dynamics algorithm 
following the ideas of Ref. 17[. The stability of the IRSB subspace can be verified by 
monitoring a suitable norm of the e^'s, e.g. 

NI^;^El''l (13) 

1 = 1 

and checking whether ||e|| ^ or not. An alternative (and numerically preferable) proce- 
dure is the following. After each sweep in the population dynamics algorithm the e^'s are 
renormalized: ^ Cj/A. The parameter A is chosen such that ||e|| = 1 is kept fixed (this can 
be done because Eq. (fTTj) is linear in the e^'s). We keep track of A and (eventually) average 
it over many iterations. The IRSB subspace is stable if |A| < 1. 

In Tab.UTlwe report the value eQ{T) of the internal energy on the marginally stable states 
(|A| = 1) for the model (jH)) with p = 3 and several values of /. As for the the dynamic and 
static energies, e^iT) and es(T), we used populations of size Af = 10^ ^ 10^ and averaged 
over 200 -v- 400 iterations. 

One further entry in Tab. |n] is the iso-complexity energy eic(r). The corresponding 
parameter mic(T) is defined by the condition 



ST(mic(T)) = T.T^{md{Td)) , 
8 



(14) 



c = 4 


eJT) = -1.21771 + 0.06409 + 0.5787 e'^''^ 

ed(r) = -1.15267 - 0.05816 T + 0.08068 + (-0.59409 + 0.65230 T) e'^^ 
eG{T) = -1.16667 -0.04109r + 0.10430 r2 + (-0.63535 + 0.47338 T) e'^/? 

ei^(T) = -1.17565 + 0.14471 e'^f^ - 0.85310 e'^f^ + 33.618 e-^^ 


c = 5 


eJT) = -1.39492 + 0.0994375 e'^/^ + 0.842632 e"^^ 

ed(r) = -1.32540 - 0.03884 T + (1.79894 - 3.17138 r)e-2/5 + 13.2849 6"^^ 
e^iT) = -1.35728 + 0.011316 T + 0.05804 + (-0.43749 + 0.29550 T) e^^/? 
eie(T) = -1.35122 + 0.13262 e^^/? + 1.4556 e^^^ 


c = 8 


e,(T) = -1.80920 + 0.07186 e'^/^ + 0.10056 6"^^ + 2.9090 e'^^^ 

ed(T) = -1.70826 - 0.04277T + (1.0528 + 0.3575 T) e"^^ 

eciT) = -1.77828 - 0.00159T + 0.07624T2 + (-2.7278 + 1.3729r) e"^^ 

eic(r) = -1.76036 + (1.0636 + 0.2779 T) e"^^ 



TABLE II: The static (cg), dynamic (ea), marginal (cg) ^i^d iso-coinplGxity {g-iq ) energies as ob- 
tained through a population dynamics solution of the IRSB equations. We present here a compact 
representation of these curves as polynomials in T and e~^^. The uncertainty due to statistical 
fluctuations of the population dynamics algorithm is Ae < 0.0005 for c = 4, 5 (and Ae < 0.001 for 
c = 8). For eic(T) the systematic error due to the uncertainty on S(j(Td) has to be added. 

where, with an abuse of notation, we used the symbol St(-) to denote the complexity as a 
function of the breaking parameter. The relevance of this curve to the issues introduced in 
Sec. IHwas suggested in Ref. |3] and will be discussed in the next Section. 

III. NUMERICAL RESULTS 

We simulated the model © for p = 3 and c = 4, 5, 8. In order to investigate the question 
in Eq. Q, we measured the internal energy of the system, O = Ti.{a), both during the 
relaxation after a quench and during a slow cooling. 

The system is always prepared in a random configuration (T = oo). In the quenching ex- 
periments the system is let evolve directly at the final temperature Tf . In the cooling experi- 
ments we used the following schedule for the temperature: ncooi Monte Carlo sweeps (MCS) 
are performed at each temperature T G {Tmax, ^max— AT, Tmax— 2AT, . . . , 2AT, AT, 0}. Val- 
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FIG. 2: Cooling experiments for several values of the connectivity c and of the cooling rate ricooi- 
Dotted lines represent the numerical results for ecooi(^i ^^cooi)- Dashed lines correspond to the 
analytical result for the equilibrium energy, and solid lines to the paramagnetic energy (for T > T^) 
or to the iso-complexity energy eic(T) (for T < T^). 

lies for Ticooi ^11^6 10^, 10^ 10^ and 10^ while those for T^^^ and AT are reported in the two 
central columns of Tab. HI 

In order to minimize finite size effects, we considered samples of size = (4 x 10^ — 1) for 
quenching experiments, and = (10^ — 1) for cooling experiments. We checked our results 
simulating few samples of size = (10^ — 1). The number of samples varies from = b 
for quenching experiments to > 10 for cooling experiments. 

In Fig. 121 we show the internal energy of the system as a function of the temperature 
during the cooling experiments with 4 different values of ncooi for each connectivity (upper 
dotted lines). It should be clear from this picture that the system is undergoing a dynamic 
arrest preventing it to reach the thermodynamic energy (lower dashed line). We also show 
in this picture the iso-complexity energy (bold solid line). Numerical evidences, to be shown 
below, strongly suggests that the system follows the iso-complexity line when cooled infinitely 
slowly. 

In order to better study the ricooi oo limit we choose one connectivity, e.g. c = 5, and 
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FIG. 3: A closer look at the cooling experiments for connectivity c = 5. 

we zoom on the interesting region. Figure El shows coohng energies for c = 5 on a different 
scale (ncooi = 10^ has been omitted for clarity). Statistical errors are smaller than symbol 
size. For comparison we also plot (from top to bottom) the analytic curves corresponding to 
the threshold or dynamic energy, ed(T), the marginal energy, eG(T), and the iso-complexity 
energy, eic(T), reported in Tab. HH 

It is expected that energy relaxation converges to a threshold energy, lying between eciT) 
and ed(T). This belief is based on the fact that for energies below eQ{T) IRSB states are 
stable. On the contrary we clearly see from Fig. Elthat a cooling experiments may bring the 
system to an energy below eG{T). In Sec. II VI we will argue that this numerical observation 
is not in contrast with the existence of well defined IRSB states. 

Next we ask whether for very slow cooling rates, i.e. for ncooi oo, the energy of the 
system follows the iso-complexity energy. In order to investigate this point we extrapolate 
the cooling energy for ncooi — > oo as follows. For any temperature T we fit the data to the 
law 

ecooi(T, ncooi) = ecooi(T) + a n~^^^ . 
For all the connectivities considered (c = 4, 5, 8) and for temperatures T < Td typical values 
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FIG. 4: Extrapolation of the cooling experiments to infinitely slow cooling rate. The crosses with 
error bars in the main frame represent the extrapolated data. Error bars include the uncertainty 
in the analytical calculation of eic(T). In the inset: difference between the extrapolated data and 
the analytical result for eic(T), for connectivities c = 4,5,8. 

for the best fitting b parameter are in the range 0.3 ^ 0.36. 

The extrapolated energies ecooiiT) for c = 5 are plotted in the main panel of Fig. IH 
(points with errors). We have included in the error bars also the contribution due to the 
uncertainty on the analytic estimation of eic(T). Thin lines are cooling energies with ricooi = 
10^, 10^, 10^, 10^, while the thick line is the iso-complexity energy. 

Within the statistical error ecooiiT) and eic(T) perfectly coincide, as can been better seen 
in the inset of Fig. HJ where the difference CcooiiT) — eic(T) has been plotted for all the 
connectivities. 

Let us now consider the energy relaxation after a sudden quench to temperature T. The 
temperatures studied for each connectivity are written in the last column of Tab. IH Each 
simulations has been run for 10^ MCS. We assume that on late times the energy relaxation 
may be approximated by a single power law behavior 

fc-quench fc-quench 

(T) + a' t-' . (15) 
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FIG. 5: The energy extrapolated from the quenches (main panel) and the best fitting b' parameter 
(inset), for c = 5 and T = 0.6. Each analytical energy is shown with 2 horizontal lines corresponding 
to its mean plus or minus one standard deviation. 

Best fitting values for h' are typically around 1/3, for all the connectivities and the temper- 
atures studied. 

The extrapolation of equench(^) depends of course on the range of times used in the fitting 
procedure, because of sub-leading corrections. In order to have an appreciation of this effect, 
we fitted the equenchiT.t) data to expression (P3| within the interval t G [tmin, ^max], with 

^max ^ 10 and tmax/'^min ^ 200. 

In Fig. Elthe extrapolated energy equench(r) (in the main panel) and the best fitting h' (in 
the inset) are shown as a function of tmax for c = 5 and T = 0.6 (very similar results has been 
obtained for the remaining connectivities and temperatures). For each value of tmax, different 
points correspond to different values of tmin- Statistical errors (not shown for clarity) are of 
the same order of the spread of points. Results seems to be almost tmin-independent, and 
also the dependence on tmax is rather mild. 

From Fig. we conclude that the extrapolated energy value is reasonably robust. More- 
over the comparison with analytic energies supports the conclusion that equcnch(r) stays 
above the energy e\c{T) reached by cooling experiments. 
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On the other hand we expect equench(^) to coincide with the FRSB dynamic energy 
ed.FRSBlT"), i.e. with the maximal energy such that the complexity is strictly positive, once 



FRSB effects have been taken into account 



12l |. While we know that edT) < ed,FRSB(7') < 



ed(T), we do not yet have any estimate of this threshold. Verifying this theoretical expecta- 
tion is therefore very difficult. Based on the data of Fig. El we guess that, if our expectation 
proves correct, then ed,FRSB(r) is 'close' to ed(T) in the models at hand. The vicinity of 
the energy extrapolated from the quenches equench(7') to the dynamic energy ed(T) is well 
verified for all the connectivities and the temperatures studied in this work (see last col- 
umn of Tab. H}. This is presumably a general property of the model (jHl) for the range of 
connectivities considered in this paper. 



IV. DISCUSSION 



We think that previous pages provide solid support for three statements concerning the 
general question (i) ecooi(T) < equench(T); (m) equench(T) > eciT)] {in) ecooi(T) = eic(T). 
For certain values of the temperature also a fourth statement holds: (iv) ecooiiT) < eciT). 
The first of these statements, in particular, implies a negative answer to the question Q. 
While the second statement was ;a,e„ expected 0. the fo..K one comes as a surprise, 
being in contradiction with some widespread expectation in the field. Notice moreover that 
these statements gets sharper as the temperature is lowered. In fact the separation between 
ed(T), eic(T) and edT) increases at low temperature, cf. Fig. El 

As for the third one, namely ecooi(^) = eic(T), there exists a simple argument which helps 
to understand this relation. Consider a system with IRSB, and assume that no temperature 
chaos is present. In other words, pure states can be traced in temperature without crossings 
or bifurcations. It is then easy to show [3| that the energy of a pure state, regarded as a 
function of the temperature, follows an iso-complexity line (the value of the complexity on 
this line depending upon the state). In particular, eic(T) is the energy of the first metastable 
states to appear when the temperature is lowered across Tj. Therefore, as far as single-time 
quantities are concerned, the system behave as if it equilibrated within the first metastable 
states encountered in its cooling history. Of course this picture is too simplified, and in fact 
the system ages also if it is slowly cooled across j^]. A pictorial view of the evolution of 
the system during a slow cooling given in Fig. ^ 
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Coming back to the discussion of metastable states in Sec. HI the present analysis suggest 
a general (and testable) strategy for tailoring out-of-equilibrium ensembles of the form 
The general idea is to add external sources Ai, A2, etc conjugated to extensive observables 
Oi, O2, etc. Such observables could be, for instance, the energy or the magnetization. The 
important rule is that any slow change in the control parameters induces an iso-complexity 
change in the state of the system. This is analogous to entropy conservation for adiabatic 
transformations in classical thermodynamics. Although here the system is in contact with 
a thermal bath, slow degrees of freedom are effectively thermally isolated. 

Let us finally comment on the relevance of our results for the analysis of local search 
algorithms for random combinatorial optimization problems (here we make the usua 



iden- 
to 



tification between cost function and Hamiltonian) . It has become customary 
compare the asymptotic cost esearch(^ — 00) achieved by such algorithms with the IRSB 
threshold energy e^iT = 0) . Here (see also 0| for a discussion concerning this point) we 
notice that: 

1. In general, the local search algorithm does not satisfy detailed balance with respect 
to the Boltzmann distribution at any temperature T. If this is the case, the very 
existence of a relation between esearch(^ — * 00) and any thermodynamic quantity [and 
a fortiori e^iT = 0)] is an open problem. 

2. In the simplest case, the local search algorithm satisfies detailed balance at any time t 
with respect to the Boltzmann distribution at temperature T{t), with T{t) a determin- 
istic schedule such that T(t) as t 00. Surprisingly, even in this case e^iT = 0) 
is not relevant. However, our results seem to imply that esearch(^ — ^ 00) > eidT = 0). 
This is an encouraging remark for the application of such algorithms. First of all 
eic(O) < ed(0). Moreover one could imagine constructing more complex annealing 
paths using clever deformations of the cost function to define the Hamiltonian. A 
smart deformation would probably allow to reduce eic(O). 
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APPENDIX A: SOME FORMULAE FOR THE DILUTED p-SPIN MODEL 

For the sake of self-containedness we report here some explicit formulae for the diluted 
p-spin model (P). These formulae can be used in computing the IRSB stability threshold 
along the lines of Sec. HIl 

It is convenient to express the cavity equations in terms of two functional order parameter 
Q[p\ and Q\p\. The first one is related to the distribution of cavity fields when one interaction 
term is removed from the system. The second one corresponds to the distribution of cavity 
fields when one spin is removed from the system. 

The 2RSB cavity equations read 

^t-"] = ^ / n^^t/^^] 4pi.---. Pir'"^' 5 [P - Po[pi, ...M. (Al) 
i=l 

/p-1 
l[dQ[pj]Ej6[p-MJ;Pi,---,Pp-i]] , (A2) 

where Ej denotes the expectation with respect to the random variable J which takes values 
+ 1 or —1 with equal probability). The mappings po[- • • ] and po[. . . ] are defined below: 

Po{x) = j l[dpM w{{y,}r^ 6{x-y, yi) , (A3) 

Poiy) = J JJ^'^Pjl^^j) ^ ~ 'p atanh[tanh/?Jtanh . . . tanh . (A4) 
The reweighting factor is 

^fa......,)- yf^;;f' . (A5) 

11^=1 2 cosh 

APPENDIX B: THE LARGE CONNECTIVITY LIMIT 

The recipe we proposed for computing the stability condition of a diluted spin glass model 
does not necessarily capture the most relevant instability. In this Appendix we show that, 
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in the large connectivity limit I oo, our approach yields the replicon instability already 
computed in This provides an important check of our calculation. 

When adapting the basic recursion (jllj) to the cavity equations reported in the previous 
Appendix, it is necessary to use two distributions Q{x,e) and Q(y,e). These distributions 
can be used to define the functions e(x) and e{y) as follows 

e(x) = j de Q{x, e) e , e{y) = j de Q{y, e) e . (Bl) 



It is easy to show that Eq. (fTTj) implies a recursion of the type [e(x),e 
where: 



1-^ \e'(x), e' 



e'(x) = jdy M{x, y) e{y) , e'{y) = j dx M{y, x) e{y) . (B2) 
The kernels of this mapping are given in terms of the IRSB solution p*(x), p*(?/): 

M{x,y) = — Yl^p,{yi) w{y,yi. . .yi^i)"" 5{x - y - yi y^.a) , (B3) 

^* i=i 

M{y,x) = J Y\_dp^{xj) A(x, {xj}) 5 — — atanh[tanh/?Jtanh/?a: • • • tanh/3xp_2]^ )(B4) 



where = 2;[p* . . . yO,,] and 

tanh /3 J(l — tanh^ j3x) YYjZl tanh jSx 



A{x, {xj}) = {p-l) 



(B5) 



1 — tanh^ /3 J tanh^ /3x Y[^=i tanh^ f]xj 

Notice that we do not need to average on the sign of J in Eqs. ()B4|) and ()B5|) because the 
distribution p^:{x) is symmetric. 

The stability of the fixed point e{x) = 0, e{y) = under the recursion ()B2|) . can be 
determined by diagonalizing the composition of the kernels ()B3|) . ()B4|) . As we will see 
shortly, this diagonalization becomes considerably easier in the limit I ^ oo. 

First of all, we rescale the coupling strength by setting J = \/p/2l in such a way that 
the energy remains finite in the large connectivity limit. We also rescale the cavity fields 



defining y = y (hereafter we shall drop the tilde). It is easy to see that, in the large 

connectivity limit, the IRSB fields distributions become 

p,(x) = 1 (cosh/5x)"^ e-"'/2A^ (B6) 
Z, 

/p-i 
]^rfp*(a;j) — tanh ■■■ tanh /3a;p_i) . (B7) 
i=i 
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The parameter A must be found by solving the equation 

A = I ^y"rfp,(x) tanh^/5x . (B8) 

This equation was already found in Ref. jl^ while solving the fully connected p-spin model. 
Taking the / oo limit also in the kernels ()B3jl and ()B4jl we obtain 



P 

M{x,y) = - p,{x) 



(B9) 



Miy,x) 



/ ]^'^P*(3^i) D{x, {xj}) 5{y — tanh/?x- ■ ■tanh/?Xp_2) , (BIO) 

7 = 1 



with 



D{x, {xj}) 



p-2 



j3{l — tanh^ j3x) JJ^ tanh f3xi 



(Bll) 



It is evident from Eq. ()B9|) that there is a unique non-vanishing eigenvalue, and that the 
corresponding eigenvector has e(x) oc p*(x). A little thought shows that the eigenvalue is 



A = - l)/52gP^2 j (cosh/5x)-^ , 



(B12) 



with p*(x) given by Eq. ()B6|) . As anticipated, this result coincides with the replicon eigen- 
value of pj]. 
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